Many current medical, manufacturing and inspection practices rely on relating three-dimensional geometry to two-dimensional projection or tomographic images. An example of a projection image is a plain radiograph, in which electromagnetic energy transmitted and refracted through or by physical objects, until film or another process, such as a digital process, creates an image. Examples of tomographic imaging technologies include, but are not limited to, computed tomography, magnetic resonance imaging, ultrasound imaging, positron emission tomography, and single photon emission computed tomography. An example of a tomographic image is an ultrasound image, in which acoustic energy transmitted and refracted through or by physical objects, until a process, such as a digital process, creates an image.
An image is information that represents a two-dimensional projection as carried by an appropriate medium. An image file stored on a floppy disk, hard disk or other storage medium is an image, as is an x-ray film. An image can be captured by processes such as beam intensity absorption, phase distortion, and frequency modulation, among many possible physical processes. The energy used to create the image can be electromagnetic, electric, magnetic, acoustic, or any other energy.
Three-dimensional geometrical knowledge derived from two-dimensional images can be used for many purposes, including diagnosis of state of health, guidance of objects that are tracked relative to the derived three-dimensional geometry, and quality control. In some of these applications, such as inferring the safe volume for drilling a bone, expert knowledge of the objects being imaged is required to infer the three-dimensional geometrical structure. For the purposes of object guidance a means of representing the derived geometry is required, as is the ability to reference the derived geometry to a coordinate frame in which the pose(s) of the object(s) that are tracked are known. A deficiency in existing use of two-dimensional images is that the derivation of three-dimensional geometry, and useful inferences therefrom, rely principally on the skill of the observer and the derivations and uses are not easily computed by automatic means.
One common use of three-dimensional geometry derived from multiple two-dimensional images is the use of fluoroscopic imaging in orthopaedic surgery. In this case, one or more two-dimensional fluoroscopic images are taken for the purpose of inferring the patient's anatomy in three dimensions; the mental image of the anatomy can be used to determine the placement of cuts, drill-holes and other anatomical modifications that a surgeon may wish to carry out. A plurality of two-dimensional images are often taken during the course of drilling or cutting the patient's anatomy to ensure that the drilling, cutting or other process is following the intended course and is not in danger of affecting any but the intended anatomical structures. At least two problems arise from taking a plurality of fluoroscopic images during the course of operating on a patient. The patient and the surgical team are exposed to additional X-ray radiation beyond the radiation needed to create the initial image. The fluoroscope can physically obstruct the surgeon who is performing the operation.
Methods exist to alleviate the two problems of additional radiation and physical obstruction. For example, the patient's anatomy can be tracked by physically attaching an object that can be detected by a computer via a tracking means and attaching a distinct object to the fluoroscope that can also be detected by a computer via a tracking means. The pose of the X-ray source of the fluoroscope with respect to the imaging system of the fluoroscope can be determined prior to image creation or after image creation. Thereafter, if a tool, such as a surgical instrument, has attached to it a distinct object that can also be detected by a computer via a tracking means and that is attached to the tool in a known manner, then an artificial image of the tool as it would appear in the fluoroscopic image of the patient can be computed. Typically but not necessarily, the artificial image of the tool is generated by determining how specific known points of the tool would appear on the fluoroscopic image had the tool been present during the physical process of creating the image, and superimposing a simple two-dimensional geometrical object on the actual fluoroscopic image. Such a method and apparatus permits a practitioner, who is a surgeon or some other user of the fluoroscope, to create one or more images of the patient's anatomy, remove the fluoroscope from the immediate vicinity of the practitioner and the patient, and observe the artificial image superimposed on the real fluoroscopic image.
At least one deficiency of the above method is that three-dimensional guidance from multiple two-dimensional images is inherently difficult because multiple two-dimensional images may be misinterpreted. An example of this problem in projection imaging is shown in the attached figure: in both two-dimensional images the tool appears to be inside the sphere, when the tool has actually penetrated through the sphere's surface. Alternate apparatuses and methods for the inferring, representing and manipulating three-dimensional geometric objects from multiple two-dimensional images are desirable.
As an illustrative example, consider the case of an orthopedic surgeon who seeks to repair a fractured hip by drilling the proximal femur from the greater trochanter through the femoral neck into the femoral head. A skilled practitioner might observe an image captured at one angle with respect to the principal axis of the femur and observe an image that is captured at some other angle with respect to the principal axis of the femur. By drawing on knowledge of anatomy, the practitioner might determine how to position a drill so as to pierce the desired anatomy and not pierce other anatomy. The observation of other fluoroscopic images, captured as the drill progresses through the anatomy, aids a skilled practitioner in fluoroscopically guided drilling. The practitioner would require considerable prior understanding of normal and pathological anatomy, and might be inconvenienced by the physical presence of the fluoroscope during the drilling process.
A practitioner who takes great advantage of existing systems might cause the creation of the two aforementioned images by means of a computer system that can create, from each image, a calibrated, geometrically corrected image. The computer system might then track the drill and a bone of the patient by real-time tracking and determine the pose of the drill with respect to an anatomical coordinate frame. The computer system might further superimpose on the calibrated, geometrically corrected image a point, line, or other two-dimensional geometrical shape that provides the practitioner with an indication of where the drill might have appeared in the calibrated, geometrically corrected image had the drill been physically present during the physical process of image creation. Thus, the practitioner might observe the computer-generated images and observe where the drill might be with respect to the femoral anatomy. Such a practitioner would not be exposed to X-ray radiation emitted by the fluoroscope during the process of image creation, and could have the fluoroscope removed from the immediate vicinity so as not to be inconvenienced by the physical presence of the fluoroscope during the drilling process.
Due to projective geometry, the actual three-dimensional relationship of three-dimensional objects may not be apparent from two-dimensional projective or tomographic images. A mathematically simple example can be constructed by considering a unit sphere centered at the origin of a coordinate frame, a vector directed from the origin in the direction (1,1,1) with length (2/3,2/3,2/3), an orthographic projection of said unit sphere and said vector to the XY plane of said coordinate frame, and an orthographic projection of said unit sphere and said vector to the XZ plane of said coordinate frame. In the XY projection the tip of said vector will appear to lie within the circle that is the projection of said sphere, and similarly for the XZ projection. Simple calculation shows that the length of said vector exceeds the radius of said sphere, so although the projections suggest that the vector lies entirely within the sphere such is not the case. For said examples of a skilled practitioner using only fluoroscopic images and of a skilled practitioner using computer-generated images, it is possible that the practitioner might pierce the femoral head and thereby cause vile undesired consequences to the health of the patient. Similar problems occur with tomographic images because two-dimensional projection images and two-dimensional tomographic images are not adequate representations of three-dimensional geometry.